MSR14 Comparisons of Encoding Techniques for Categorical Features in Linear Regression Models

In healthcare research, data contains many types of categorical variables, such as race, country, zip code, from low to high number levels in each category. However, aforementioned features need encode into numeric forms before applying machine learning algorithms. Therefore, it is critical find a suitable encoding method for coefficient estimation and prediction. this study, we investigate three commonly used methods compare them estimation, feature selection, prediction linear regression analysis. We label encoding, one-hot target leave-one out (n_level=5) (n_level=50) variables under balanced unbalanced synthetic designs. apply different algorithms (ordinary least squares (OLS), Bayesian ridge logistic regression) on datasets the binary classification settings. low-level settings with continuous outcomes, all can identify true important features, achieves smallest mean absolute error (MAE) both coefficients For classification, fails detecting accuracy around 50%. OLS scenario, derived shift far away value, especially imbalanced high-level (n_level=50), outperforms other two MAE, stable selection. One-hot has relatively however, could not estimations are stable. Label-encoding able largest MAE. Target leave one traditional terms performance, categories.